Skip to content

Conversation

@germasch
Copy link
Contributor

The idea behind this is good, since adios2py and xarray-adios2 are independently useful. The execution is horrible, though,
in that this is a unreviewable mega PR.

Basic testing succeeds, though, and whatever rough edges there are, they shouldn't be too hard to fix up.

In existing code, it'll be necessary to replace direct reading of a psc adios2 file with something like

def _open_dataset(filename: os.PathLike[Any]) -> xr.Dataset:
    ds = xr.open_dataset(filename)
    return pscpy.decode_psc(
        ds,
        species_names=["e", "i"],
        length=[1, 12.8, 51.2],
        corner=[0, -6.4, -25.6],
    )

via lambda doesn't seem to make much of a difference. In both cases,
when doing just a selection, it'll be kept lazy, but say rescaling by
multiplying by a scalar leads to the data being loaded.

Probably something where dask comes in.
Since the backend is registered by means of pyproject.toml now
No need to have a psc.RunInfo until later
This is all about gettings things more in line with other xarray backends
In particular this requires the subtlety of actually returning
xr.Variable from open_store_variable(), not xr.DataArray
keep compatibility with files missing this information, though, even
though there's not much we can do, other than special casing psc so
that at least doesn't break
however, keep current behavior, ie always reverse
but also keep reversing for pfd* tfd* to maintain compatibility
This isn't consistent with xarray typing, and easy enough to avoid
by sticking with open_dataset(Adios2Store.open(group))
Looks like just something left over from the template
We might not want to do this in the long run, since this leads to a
transitive dependency on adios2, which is binary and potentially iffy to
install, so if one were to want to use pscpy without the adios2 reader,
that could cause some hassles.
@codecov
Copy link

codecov bot commented Jan 14, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Files with missing lines Coverage Δ
src/pscpy/__init__.py 100.00% <100.00%> (ø)
src/pscpy/psc.py 98.03% <100.00%> (+15.11%) ⬆️

@germasch germasch merged commit 2bf2fc1 into psc-code:main Jan 15, 2025
4 of 5 checks passed
@germasch germasch deleted the pr/pscadios2 branch January 15, 2025 03:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant